DARL: Distance-Aware Uncertainty Estimation for Offline Reinforcement Learning

To facilitate offline reinforcement learning, uncertainty estimation is commonly used to detect out-of-distribution data. By inspecting, we show that current explicit estimators such as Monte Carlo Dropout and model ensemble are not competent provide trustworthy in learning. Accordingly, propose a non-parametric distance-aware estimator which sensitive the change input space for Based on our new estimator, adaptive truncated quantile critics proposed underestimate samples. We able offer better compared previous methods. Experimental results demonstrate DARL method competitive state-of-the-art methods evaluation tasks.